Reassessing ‘The Effect of the Seattle Police-Free CHOP Zone on Crime’

A counterfactual critique

Charles C. Lanfear

University of Cambridge

The CHOP

2020 CHOP timeline

 

Piza & Connealy (2022)

 

Their question

 

Did SPD’s withdrawal increase crime?

 

The geographically focused, time limited nature of CHOP provides an opportunity for a natural experiment to test the effect of the autonomous zone on crime. Given the context of CHOP, this study may further be considered a test of police abolition, the most radical proposal advanced in the police defunding movement. (Piza & Connealy 2022:36)

Piza & Connealy’s strategy

 

Microsynthetic control approach:

  1. Treat police withdrawal as quasi-experimental treatment
  2. Match 36 “treated” street segments1 to similar untreated “controls” among >20,000 candidate segments
  3. Compare crime counts between treatment groups

Matching variables

Continuous:

  • 6-day period police-recorded crime counts (the only time-varying measure)
  • Count of commercial businesses
  • Count of consumer-facing facing establishments
  • Block group crime density
  • Length of the street segment

 

Binary:

  • Principal vs arterial street
  • If pre-CHOP total crime > 80th percentile
  • If police beat > 80th percentile of police calls
  • In block group with > mean disadvantage
  • In block group with > mean non-white residents, residents 15-29, vacant homes, and owned/rented homes

Their findings

Results indicate crime significantly increased in the CHOP zone, the encompassing two-block area, and the overall East precinct service area.

The current study found comparatively stronger effects than the general literature on depolicing. The significant crime increase is particularly noteworthy given the short time frame of the CHOP occupation and retreat of police from the area theoretically making it more challenging for crimes to be reported by citizens and/or proactively discovered by officers. This suggests that police abolition, the most extreme form of the police defunding movement, may significantly compromise public safety. (Piza & Connealy 2022:35–36)

Counterfactuals

Potential Outcomes


image/svg+xml A – = Some unit Treatmenteffect Y 1A Y 1A Outcomeif treated Y 0A Y 0A Outcomeif untreated

Ignorability


image/svg+xml A B If A islike B Y 1A Y 0A Y 0B Y 1B = Y 1A Y 1B = Y 1A Y 1B – = Treatmenteffect Unobserved Y 1A Y 0B

What is our counterfactual?

 

Causal inference hinges on having the right counterfactual.

 

What if we don’t know what \(Y_{A}^{0}\) we want?

What if the \(Y_{B}^{0}\) we have is not similar to \(Y_{A}^{0}\)?

 

This is Piza & Connealy’s problem

Piza & Connealy Revisited

Implicit causal model

G chop CHOP Zone crime Crime chop->crime place Measured Place Characteristics place->chop place->crime protest Protest protest->chop protest->crime

Assumptions:

  • Primary: Protest is independent of CHOP Zone and/or Crime
  • Secondary: No unmeasured Place Characteristics are relevant

Problems

  • Protest definitely caused CHOP Zone and probably caused Crime
    • It is not conceivable the police pullout would have occurred absent the large-scale protest and repeat conflicts with police
    • The police pulled out because clashes with police were escalating
    • Protest caused massive increase in ambient population
  • Measured Place Characteristics don’t explain why Protest or CHOP Zone occurred
    • Places with similar characteristics–including crime–did not have large-scale protests
    • Capitol Hill has unique social-ecological characteristics associated with protest

Protest Capitol Hill

Typical Capitol Hill

Capitol Hill is where protests happen

 

Seattle’s protest stood out from the national activity due to their extreme magnitude and duration, particularly1 in the context of the Capitol Hill neighborhood. (Piza & Connealy 2022:36)

Capitol Hill is an unusual area of prone to protests and conflicts with police

Crime spiked in Capitol Hill absent police withdrawal on July 25th, 2020 due to another protest.

What is their counterfactual?

Given these issues, what are Piza & Connealy estimating?

 

\(Y_A^1\) they observed

How many crimes occurred after police withdrew to de-escalate conflict with an existing large-scale protest against police

\(Y_B^0\) they estimated

How many crimes would have occurred if neither the protest nor the withdrawal occurred1

 

Critically, the difference between these does not tell us if withdrawing from the East Precinct increased crime

What counterfactual did they want?

Piza & Connealy’s (2022) counterfactual is only implied:1

A more prudent solution [than withdrawing] would have been to more effectively engage with protesters to quell the violent activity. (52)

Implied \(Y_B^0\): How many crimes would have occurred if police had remained but de-escalated the conflict

But recall SPD withdrew to de-escalate after exhausting options:

SPD abandoned the precinct in an attempt to quell the confrontations, resulting property damage, and injuries to both police officers and protestors. (36)

Police-reported crime in CHOP zone is in large part due to police activity

Much larger “treatment effect” if we date treatment at start of Cap Hill protests than start of CHOP

Plausible counterfactuals

Even if they could estimate it, their implied counterfactual is implausible

But other realistic counterfactuals exist:

Alternative \(Y_B^0\): How many reported crimes would have occurred had…

  • SPD continued battling protestors?1
  • SPD suppressed the protest?
  • The national guard been deployed?2

Summary

Design

  • Theory needs to be correct
  • Know your counterfactual
  • Other cities may offer valid counterfactuals (e.g., Portland)

Substance

  • Estimate does not correspond to any useful estimand
  • “CHOP effect” is not likely to generalize

Feedback and Questions

Contact:

Charles C. Lanfear
Institute of Criminology
University of Cambridge
cl948@cam.ac.uk